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ABSTRACT 

The third in a series of international conferences on 
measuring quality in post-secondary education was held to design four 
proposed project briefs, to identify new areas of collaboration, to 
encourage participation in the current projects, and to promote 
continued dialogue and exchange on the projects. Each of four working 
groups addressed one of the following four projects: (1) 
international assessment of student learning in specific disciplines; 
(2) impact of quality assessment on higher education systems; (3) 
drop-out, retention and enrollment patterns of students; and (4) the 
role of peer review in evaluation and assessment. V^orking Group 1 
suggested modifications to the project's design, determined the focus 
of the project as a comparison at the program level, and acknowledged 
the need for funding and the complexity of the work. Working Groups 2 
and 3 outlined the purpose of their study, methods, and funding; and 
chose the participating countries. Working Group 4 planned a two 
stage project in which stage 1 will consist of an inventory of forms 
of peer review in operation in a substantial number of countries and 
stage 2 will investigate the legitimacy of peer review. New projects 
were proposed in a comparative study of graduate outcomes. Appendix 1 
lists part^* cipants and their affiliation; further appendixes contain 
the reports from each of the working groups. (JB) 
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Introduction 

Following on from the first two working conferences during which proposals for four 
specific research projects had been identified for collaborative work, the purpose of 
this, the th rd, international gathering was: 

(a) to move the four projects forward by designing fully formed project briefs; 

(b) identify new ideas/areas that people wish to pursue collaborative research on, 

(c) establish interest/willingness to participate in the current projects; 

(d) provide a means of continued dialogue and exchange of ideas on the four 
projects and any new projects; 

The conference was attended by researchers from Australia, Austria, Belgium, Finland, 
Germany, Hong Kong, Mexico, the Netherlands, Norway, South Africa, Sweden, 
United Kingdom, and United States of America. 

Each of the participants joined one of four working groups (representing the four 
agreed collaborative projects): 

Working group 1: international assessment of student learning in specific disciplines 

Working group 2: impact of quality assessment on higher education systems 

Working group 3: drop-out, retention and enrolment patterns of students 

Working group 4: the role of peer review in evaluation/assessment process 

As the aims of working groups 1 and 4 overlapped to some extent, the two working 
groups were combined during the conference. 

A full participants' list, including working group representation, is attached as 
appendix 1 to this report. 

Reports from the four working groups 

Working group 1: International assessment of student learning in speciHc 
disciplines 

The basis for discussion of this working group was a project proposal which was 
formulated at a meeting held prior to this conference in Stockholm 26-27 March 
1992. During this meeting the design and outcomes of two research projects on 
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international comparisons of study programmes were discussed, namely the '*EQ 
project" undertaken jointly by CNAA, CHEPS and HIS\ and the business 
administration comparative project initiated by the Swedish National Board for 
Universities and Colleges^. 

The working group suggested some modifications/amendments to the design of the 
project. These have now been incorporated into the final version of the project design 
which is attached as appendix 2 to this report. In addition, it was agreed to change 
the title of the project to International Comparison of Study Programmes, 

The working group spent a substantial amount of time discussing the focus of the 
project: whether it should be directed at a comparison of study programmes, student 
achievements or a combination of both. It was agreed that in order to keep the project 
workable/realistic, the focus should remain at a comparison at the programme level. 

It was acknowledged that the project, as outlined in the project design, would be a 
difficult and complex exercise. A major problem would be to attract sufficient 
funding. The costs involved in conducting such a project are substantial both because 
of the design and length of time it would take (approximately 18-24 months). With 
respect to strategies that might be adopted to raise sufficient funds, it was agreed to 
disseminate the outcomes of the predecessor projects (see footnote) as widely as 
possible. This is already under way. It was also agreed that diversified funding 
would be the most realistic option as no single body/sponsor was likely to be found 
to cover all the necessary project costs across all participating countries. It was agreed 
not to actively pursue funding at the present time but to await expressions of interest 
from potential funders following the dissemination of the results of the existing 
comparative projects. However, the members of the working group would be mindful 
of funding possibilities in their own countries and with supra-national bodies. Some 
developments on which the project might "piggy back" were highlighted but no 
concrete results can be showii. 

2*2 Working group 2: Impact of quality assessment, audit or assurance 

A draft research proposal, based on discussions at this conference, is attached as 
appendix 3. 



^ John Brennan, Leo Goedegebuurc, Tarla Shah, Don Westerheijden and Peter Weusthof, 
Towards a Methodology for Comparative Quality Assessment in European Higher Education: 
A Pilot Study on Economics in Germany, the Netherlands and the United Kingdom, 
London/Enschedc, CNAA/CHEPS, 1992. 



^ National Swedish Board for Universities and Colleges, Business Administration and 
Q Economics Study Programmes in Swedish Higher Education: An International Perspective, 
ERIC Stockholm, UHA, 1991, report number 18. ^ 
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23 Working group 3: Drop-out, retention and enrolment patterns of students 

A report from this working group of discussions held during the conference is attached 
as appendix 4. It outlines the purpose of the study, the methods and the funding. It 
is anticipated that a full project proposal will be completed by mid- July 1992. In the 
first instance the participative countries will be those represented during the 
conference ie Austria, Finland, Germany, the Netherlands, South Africa, the United 
Kingdom 9iid the United States of America. 

2.4 Working group 4: the role of peer review in evaluation/assessment process 

The basis of the working group's discussion was the paper on the role of peer review 
in the evaluation/assessment process prepared by John Brennan after the second 
meeting in Edinburgh (see proceedings of the second conference, pages 14-16). 

A two stage project was constructed by the working group: stage 1 will consist of an 
inventory of forms of peer review in operation in a substantial number of countries; 
stage 2 will investigate the legitimacy of peer review. Appendix 5 gives details of the 
two stages. 

3 New Projects 

Comparative study of graduate outcomes 

This study would be concerned with the measurement of outcomes, in the context of 
national goals, from having pursued a higher education programme by contacting 
students in, say, the last six to eight months of their study programme. The following 
issues would be addressed: 

- duration of time taken to complete study 

- students' academic and intellectual achievement 

- students' achievement in technical and social skills 

- students' attitude towards their school, college, university and society 

- students' satisfaction with their institution 

Michael Nettles would produce a paper/project proposal based on his own ideas and 
the discussion during the conference. Penn State (Jim Ratcliff) would take 
responsibility for distributing this document to the conference participants. 

4 Next steps 

4.1 Although u is not planned to anange a further full conference in the near future, 
participants were asked to note August/September 1993 as a possibility. 
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4.2 Individual meetings of the project groups will be arranged separately as required by 
the project co-ordinators. Possible dates are: 

6-9 September 1992 (Brussels) 

(when the next EAIR conference is to be held) 

29 October to 1 November 1992 

(when the ASHE conference is to be held) 

24-28 May 1993 (Montreal) 

(when the International Network of the HKCAA will be holding its coference) 
43 The co-ordinators for the projects are as follows: 

International assessment of student learning in specific disciplines: Leo Goedegebuure 
and John Brennan 

Impact of quality assessment, audit or assurance: Harold Silver and Don Westerheijden 

Drop-out, retention and enrolment patterns of student: Riekele Bijleveld, Michael 
Nugent and Vincent Tinto 

The role of peer review in the evaluation/assessment procesi^: John Brennan and Lynn 
Meek 

5 Secretariat Support 

Responsibility for providing appropriate secretariat back-up will be on a rotating basis 
from now on between Penn State and CHEPS, commencing with Penn State. CHEPS 
will take over in May 1993. 
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Appendix 2 



International Comparison of Study Programmes 

Design Proposal 

CHEPS 

May 1992 

1 Purpose 

The purpose of international comparison of curricula is to establish the effea of different higher 
education programmes within the same discipline on the level and breadth of knowledge and skills 
graduates are expected to have acquired after completion of their degree. 

The level of comparison should be the study programmes, not institutions (which often offer 
programmes in several disciplines), nor countries (where considerable differences exist among the 
individual programmes even withing a single discipline). Taking this programme angle, the national 
higher education system and the discipline-wide features are important boundary conditions for the 
individual programme. 

2 Indicators 

Several broad categories of indicators are possible on the knowledge and skills to be desired in 
graduates, most importantly: 

• curriculum description and comparison; 

• assessment of student (final level) work; 

• comparative tests of students or graduates; 

• graduates' opinions and achievements; 

• employers' opinions. 

The core of comparison should consist of the first two points* description and comparison of 
curricula and assessment of student work. Curriculum comparisons give insight into the what, how 
much and how of teaching in different study programmes (the independent variable according to 
the purpose). Student work gives insight into learning outcomes of the teaching offered, as well 
as insight into the operational norms and values in the study programme. Therefore, student work 
is the most important category of indicators (the dependent variable according to the purpose). 

The latter two categories, graduate and employer data, are thought to be very informative 
additional sources, but are expected to be beyond reasonable time and resource limits. 



Comparison - Design - 2 

Tests of students or graduates are deemed to be undesirable for several reasons, among them 
the fact that any one test would be too narrow in scope to cover a representative part of the level 
and breadth of skills and knowledge of students or graduates in different countries. 

3 Methods 

To assess the level and breadth of student work, experts are needed. Accordingly, the prime 
method in this project should be peer review. The expert assessment should be assisted by 
providing them with a clear focus for assessment, and informing them through struaured, 
comparable data-sets, 

3 J Experts for the Review 

The experts needed are disciplinary experts (^peers' from within or outside, depending on the 
discipline) and 'external* experts in the area of programme structure, design, etc. (Of course, 
persons can have both these charavteristics.) To avoid self-interest or too strong a bias towards one 
of the higher education programmes or systems involved in the project, experts, expecially peers, 
should be chosen as much as possible from other countries. For a sufficient level of knowledge 
about the higher education systems involved, it might be necessary to include some (other- 
disicplinary) experts from the countries from which programmes will be compared. 

The number of experts should be around seven (five disciplinary peers from different 
specialisms/sub-disciplines and two ^external* experts), maximum ten. Larger groups threaten to 
become unworkable. 

3.2 Foe ^ for Assessment 

The review should focus on objectives, subjects, content and methods of student assessment in the 
study programmes. 

Objectives: What are the general objectives of higher education; what are the programme specific 
objectives? 

Subjects: Which subjects are taught, for what duration, which proportion of the total time of study? 
Which subjects are examined (or otherwise involved in the final assessments of the sudent in 
the programme)? Focus should be on both general educational elements like conununication, 
.9^. problem-solving, etc., and pure disciplinary elements. 

^ IB 
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Comparison - Design - 3 

Contents: To what level of knowledge or skills are the subjects taught? To what level of knowledge 
or skills are the subjects examined, i.e., which criteria are applied to decide whether students 
fail or pass? 

Methods: In what way does student assessment take place (especially at the final moments of the 
study)? Which skills and types of knowledge are assessed? 

3 J Proceeding of Peer Review 

Ideally, die expert group -should have three plenary meetings, each lasting two days. 

• To discuss the information in the data-sets with each other and with the research team; 

to identify questions to be put to die programme representatives (so that they may prepare 
answers). 

• To discuss informally the questions, in sub-groups of two or three review panelists, posed 
previously with about two progranime representatives each; 

to draft the outline of the report. 

• To finalize die report. 

If necessary, meeting diree might be replaced with a written round of comments by the experts on 
the draft report. 

For reasons of efficiency and to prevent spurious expertise feelings, site visits will not be included. 
Instead, representatives of the study programmes involved will be invited to an expert group 
meeting in order to provide the experts with additional information about the individual 
programme. 

3.4 Preparation of Data-Sets 

Previous to die peer review, and aiming at support for it, data-sets have to be prepared about the 
individual study programmes. These data-sets should be: 

• clearly focused towards the object of comparison; 

• highly and consistently structured in a way evident for die user; 

• as succinct as possible. 
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As a general model for the data-sets, the 
schedule presented here is useful; data in each 
cell of the matrix would have to be filled 
according to a checklist. Data can be gathered 
by * national* research teams for each country. 
One research team would have to edit the final 
version of the reports to ensure consistent 
structuring of the data. In general, the essential 
data, i.e., the programme-level data about the 
educational process, should focus on the core ( 
the discipline under study in representing and 



Comparison - Design - 4 
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the discipline. Assistance from representatives of 
immarizing the information is highly desirable. 



The data, at least at the programme level, should lead to clear *one-liner' statements or questions 
for each relevant heading in the checklist, so as to focus the discussion in the peer review. 



As an indication of the efforts involved in carrying out a project of the above nature, a time- 
schedule of some 18 to 24 months is envisaged. 



Appendix 3 



DRAFT 



IMPACTS OF QUALITY ASSURANCE SYSTEMS 
PROPOSAL FOR A RESEARCH PROJECT 



L Purpose 

The purpose of the proposed project is to identify the impacts of quality assurance 
systems in a small group of countries, and as far as possible to explain the conditions 
in which such systems are successful and by what criteria. 

The United States does not have a national quality assurance system, although it has 
a network of quality assurance components. Apart from those within institutions or 
consortia of institutions, these components include ones for regional accreditation of 
institutions, national accreditation of professional programmes, statewide programme 
review procedures for public higher education institutions, state licensing requirements, 
and var-ous kinds of assessment procedures. Some other countries have nationally 
mandated or voluntary systems. The United Kingdom, for example, in addition to its 
long-established 'external examiner* system, has, since the mid-1960s, had the 
govenmient-created Council for National Academic Awards to act as the assurer of 
standards in non-university higher education, and from the end of the 1980s an 
Academic Audit Unit established by the universities' Committee of Vice-Chancellors 
and Principals. (The CNAA and the AAU are about to disappear, to be replaced by 
unified national quality assurance arrangements.) The Netherlands has separate 
systems of quality assurance for its universities and its institutes for higher vocational 
education. The Hong Kong Council for Academic Accreditation, established in 1990, 
advised on academic standards in five institutions of higher education and acts to 
monitor and raise standards. Other countries, including France, Mexico, Australia, 
Austria, and some of the Scandinavian countries have established, or emergent, 
systems for quality assurance purposes. 

Under the auspices of the National Center on Postsccondary Teaching, Learning and 
Assessment at Pennsylvania State University, the Center for Higher Education Policy 
Studies at the University of Twente, The Netherlands, and the Council for National 
Academic Awards, London, three meetings have been held of an International 
Working Conference on Measures of Quality in Higher Education since 1991 (in 
Washington, D.C., Edinburgh and London), bringing together representatives from 
countries mentioned above and others, and these have discussed the possibility of 
developing a comparative research project into the impacts of such quality assurance 
systems. The present proposal is an outcome of those discussions. 
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2. Quality Assurance 



The tenninology of 'quality assurance' varies between and within countries, and the 
concept is sometimes equated with quality audit or quality assessment, and to some 
extent quality control. Quality assurance is used here to encompass all those 
procedures which monitor institutions' mechanisms for securing appropriate standards, 
including the management and delivery of programmes of study, the evaluation of the 
quality of teaching and learning, and the assessment of students. 

The language and explicit processes of quality assurance have become a central 
feature of the discourse of higher education particularly in the 1980s and 1990s, as a 
result of pressures of various kinds from outside and within institutions and sectors 
of higher education. There have been, for example, persistent public demands for 
greater accountability, programme improvement, and more reliable measures of student 
performance. Institutions have sought to clarify missions and establish the means of 
making evaluative comparisons across programmes, institutions and time. The quality 
assurance mechanisms in different countries have been established in different funding 
and governance structures, different balances of public and private higher education, 
different percentages of post-secondary transition to higher education, different 
distributions of responsibilities amongst central and local govenmient and quasi- 
govemment bodies, and different patterns of academic and professional qualifications. 
A number of countries represented at the International Working Conference have 
therefore undertaken, as a first step, to describe the reason, or clusters of reasons, for 
the establishment of their quality assurance bodies and their modes of operation, and 
these country descriptions arc to be made available by September 1992. 

Although there are major differences, therefore, amongst the quality assurance bodies 
under consideration, their position in the national systems of higher education, and 
their procedures, their common feature is the intention to have an influence on the 
quality of the higher education delivered by institutions and experienced by students. 
There are different ways of measuring or observing such influence or impact, and it 
is central to this project to identify these and the criteria on which they operate. The 
core of the project is to discover what it is possible to know about the impacts , and 
the ways of knowing it. An important clement in a comparative study is the 
opportunity for national researchers to consider what they consider to be the main 
ways in which a quality assurance system has impact, and to examine the impacts and 
their original detail, but also to test out their analysis with visiting researchers from 
other countries. A comparative study is therefore intended to be not simply a set of 
parallel descriptions and analyses, but also an interwoven set of studies to highlight 
those aspects of the procedures and outcomes that can play a valid part in discussions 
in other countries, and amongst countries. 

It is against this background that the financial basis for the proposal has been shaped. 
Within each of the countries selected funds wii! be sought for the conduct of the 
essential work in that country. For example, in The Netherlands the funds will be 
sought by staff of the Centre for Higher Education Policy Studies, and in the United 
Kingdom by staff at the Open University. The central funding sought here is to 
sustain the international part of the work, enabling each visiting team (of one or two 
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people) to visit another country or countries to share in the analysis; for further 
international meetings to refine the analysis and determine the priority issues; and for 
the purposes of communication, editing and publication. It is intended that material 
produced at various stages of this project will be published and disseminated, without 
awaiting final outcomes of the research. For example, published 'working papers' 
could begin with a compendium of the national quality assurance profiles already 
being planned and produced. 

3, Project Outline 

The aim of the project can be summarised as being to explain the similarities and 
differences found in higher education systems in the extent to which quality assurance 
procedures are successful in producing anticipated effects, and in which associated 
unintended effects may also occur. The independent cluster of variables, that is, the 
cluster of explanatory factors, is to be found (1) in characteristics of the quality 
assurance system(s), and (2) in structural and cultural aspects of the particular higher 
education system and its political and other contexts, including the origins and 
objectives of the quality assurance system. 

The Quality assurance system can be seen, in quasi-experimental terms, as the 
'treatment'. Since treatments are not the same in different countries the different 
characteristics of the particular treatment can be included in the independent cluster. 
To narrow the scope of the project it is proposed to focus on a group of quality 
assurance systems with the same object - to assess the quality of teaching and 
learning in higher education. Quality assurance systems specifically or primarily 
focused on research or institutional management would not be included, and the focus 
on teaching and learning would be on an area to which most attention appears to have 
been given in introducing official quality assurance systems in recent years. Because 
of differences among higher education systems the scope cannot be limited very 
rigorously, but as long as the primary focus of the quality assurance system is on 
teaching and learning it is a candidate for the proposed project. 

The dependent variables are the impacts or effects of the quality assurance system. 
These can be anticipated and unanticipated, in the eyes of the initiator(s) or other key 
actors, ranging fi^om the govenmient or other national body, to the institutions 
themselves. Different types of effects can be distinguished, for example whether they 
are primary or secondary effe^.is, ar .d the field to which they apply, such as: faculty 
performance and development; student learning and attainment; programme 
availability, management and delivery; evaluation mechanisms; different levels of 
administration and management; resource availability and deployment; relationships 
of institutions and their sub-units to outside bodies and communities. Impacts exist 
on all levels of the higher education system. 

An international comparative study makes it possible for a large range of observations 
in these connections to be made. Within a single higher education system there is 
often only one quality assurance system, and a study of one case cannot reveal the 
influence of factors that remain constant within this case, while these may, given the 
variety that is possible for these factors, be very important for the range of possible 
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values of the dependent variables. A case might therefore be made for a 'most 
dissimilar systems design', for that would maximise the range of the independent* 
variables. What that will mean in practice for the proposed project will be clear only 
after the cluster of factors or variables that are thought to be important have been 
established. 

For the comparative part of this project it is not important to settle in advance the 
question of what is the 'impact focus', though this is of the utmost importance for the 
constituent studies, in which it is crucial to know what are the intended effects, and 
to distinguish between changes resulting from the quality assurance system, and other 
changes. In the comparison the important aim is to find out how contextual factors 
and the characteristics of the quality assurance system influence it effectiveness. In 
policy terms, the focus is not on how quality assurance reaches it intended effects, but 
how it can be organized so as to be as effective as possible. 

The kinds of questions to be addressed would include:- 

1. Is there any effect, for example - is there any increase in the quality of 
teaching programmes? Is this increase a change from the level of increase 
when there was no quality assurance system? 

2. To what extent can these effects be ascribed exclusively to the operation of the 
quality assurance system? What other changes in the context of the object of 
study may have caused these effects? 

3. Which characteristics of quality assurance systems tend to be associated with 
which types of effects? Since peer review systems, for example, have a higher 
legitimacy in the academic field do they have more identifiable effects on 
programme content than systems based on performance indicators, which may 
tend to produce more 'administrative* effects? Do newly introduced quality 
assurance systems carry a 'halo' effect? Therefore: 

4. Wha{ are the effects of the 'age' of a quality assurance system on its 
effectiveness:, and: 

5. Which other contextual factors influence the effectiveness of quality assurance 
systems? 

4. Procedures 

The proposal is to conduct the project in two stages. In the first stage the first two 
questions will be addressed in selected countries. The second stage consists of the 
comparative study (questions 3 to 5). As a preliminary, however, as indicated above, 
a number of country profiles arc being produced, describing the quality assurance 
procedures existing or emerging in a wider range of countries, and the policy and 
contextual origins of these systems. At an early stage of the project, therefore, it will 
be possible for experience of introducing and operating quality assurance, in higher 
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education systems to be shared with countries where quality assurance is either 
emerging or being contemplated. 

The most useful countries for intensive study are therefore ones where a quality 
assurance system has been in operation for a sufficiently long period of time to make 
at least primary effects visible. The countries seen as most adequately fitting this 
description are the United States, the United Kingdom, The Netherlands, France and 
Australia. The final choice will depend on confirmation by the country profiles being 
produced, but is already clear that the first three of these will need to be included. 

In comparative projects by international research teams it is common for papers on the 
individual cases to be prepared by one author or a small group of authors from the 
country concerned. The disadvantage with this procedure is that idiosyncrasies cannot 
be controlled for. While retaining the maximum amount of 'local' knowledge, 
therefore, it is proposed to have small teams of researchers from two countries 
working on each of the cases. The 'local' researcher or research group will keep the 
initiative, but be in close contact with a researcher or small research group form 
another country involved in the quality assurance study. 
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Appendix 4 



Report of Working Group 3: Drop-out, retention and enrolment patterns of students 

The purpose of this study is to investigate student participation and persistence cross- 
nationally. It is hoped that such a comparative investigation will help researchers, 
administrators and higher education policy makers to gain insight into issues which span 
across national frontiers, manifesting themselves often quite differently in relation to structural 
and cultural differences. Through the elucidation of differences and similarities in persistence 
patterns among national systems this study intends to explore new paths of research which 
promise to have practical and academic value. Up until the present, international comparative 
literature on student persistence and participation has been scarce, if not totally absent. A 
study such as the one proposed here will be unique in subject, character and scope and add 
substantially to the theoretical and practical understanding of factors contributing to drop-out 
and time to degree. 

The working group developed a comparative research framework to investigate student 
participation and persistence in postsecondary education. It was decided that a comparative 
analysis of student participation and persistence among higher education systems would be 
undertaken by first examining educational systems from the standpoint of structure and 
selectivity to determine the overall flow of students within the entire educational system. The 
second stage would be to provide an overall analysis of the higher education systems in 
relation to student participation and persistence. It is the primary intent of this comparative 
study to focus on the participation and persistence of students within first degree programmes 
in each country. Within the context of first degree programmes, longitudinal data of student 
cohorts from each country will be examined in order to construct a comparative framework. 

Structure and selectivity 

In order to determine where, how and when students are formally selected for postsecondary 
education, the education systems of each country will be examined in relation to structure and 
selectivity. For example, whereas in the USA selection can be an important mechanism to 
regulate and control the enrolment at the institutional level, any student in Germany and the 
Netherlands who completes an academic secondary degree is allowed to enrol in university 
studies. In some countries, such as Germany, formal selection occurs early in the national 
education system. In other countries, such as the United States, formal selection for higher 
education occurs much later, at the postsecondary level. Other selective factors are present 
at various levels, some of which arc deeply rooted in the tradition of the system. The 
question, therefore, pertains to what built-in filters determine the number of students 
participating in postsecondary education in each country, thus reflecting the selectivity of the 
education system regarding academic education (sec Clark, 1986). Secondary da.a sources 
will be used to determine the proportion and numbers of students who continue on to 
postsecondary education. General descriptive characteristics of the students will be listed such 
as age, gender and ethnic background. 

Description of higher education systems 

Following the description of flow of students within the education systems toward higher 
education, a comparative description of the higher education systems will be undertaken in 
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relation to student participation and persistence. Higher education systems generally house 
many different types of education. To provide an overall overview of each system, all 
possibilities of postsecondary degrees will be described in relation to structure and selectivity. 
In describing each system, the following factors will be taken into consideration: 

- different sectors of higher education 

- variants of degree possibilities 

- access and selection (discussed above) 

- institutional differentiation and hierarchies 



Types of education 

The following is a brief overview of the major degree possibilities in the countries under 
investigation. 



1 Germany 

The German higher education system is made up of traditional research universities 
(Wissenschaftlich Hochschulen) and vocational-oriented universities 
(Fachhochschulen). Before unification of the two Germanics, the Federal Republic 
of Germany had approximately 1.5 million students studying at 68 universities and 
50,000 at Fachhochschulen. The primary focus of this study will be on courses at 
universities that award Magister and Diplom degrees, or lead to the Staatexamen 
(degrees offered by the State). Selection for admission occurs at the secondary level 
upon successful completion of the Abitur (academic high school finishing 
examination). Institutions therefore have no control over admissions. German higher 
education receives public financing and charges no fees to students. 



2 The Netherlands 

In the Netherlands the Higher Vocational sector and the 14 public universities both 
with four-year courses form the two sectors of higher education. The primary focus 
of this study will incorporate all first-tier university courses leading to the 
Doctorandus, Ingenieur or Meester. As in Germany, selection occurs at completion 
of the secondary finishing exam level, rather than the institutional level. Tuition fees 
of approximately 1500 guilders are charged to students, though every student is 
entitled to a basic governmental grant. Presently, there arc approximately 170,000 
students studying at the universities. 



3 The United States 

The unique development of higher education in the United States has seen a 
proliferation of a wide variety of institutional types and sizes. This range of 
differentiation in the United States has led to classification schemes such as the 
Carnegie Classifica ton where, institutions categorised at one extreme of the scale 
grant doctorates and perform research, at the middle of the scale offer non-doctoral 
graduate study and at the other end of the scale offer only undergraduate degrees, 
some being only two-year degrees from community colleges. In addition, institutions 
Q arc classified as public or private. The primary focus of this study will be on the 
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four-year degree. There arc approximately 1200 traditional four-year institutions 
offering a bachelor of arts or science degree. Total FTE enrolment at four-year public 
and private institutions is approximately 6.8 million. Unlike Germany or the 
Netherlands, selection for American higher education occurs at the institutional level. 
Educational costs are passed on to the students through fees and tuition, which vary 
depending upon institutional type and need of student. 

Austria 

In Austria higher education is dominated by the university sector (12 public research 
universities). The non-university sector is small and is made up mainly of teacher- 
training institutions. Presently, new non-university institutions, Fachhochschulen, are 
under development. The formal selection of students is !:imilar to the German system. 
Thirty per cent of the Austrian secondary school students finish Gymnasium (the 
academic upper secondary track) and pass the Matura (similar to the German Abitur). 
This examination serves to formally select students for entrance into the universities. 
Universities, therefore, do no select their own students. Unlike Germany, Austria has 
no centralised Numerous Clausus. Everybody who has passed the Matura has "open 
access" to higher education. 

Finland 

The higher education system consists of 17 "scientific" institudons plus 3 colleges of 
arts. Until now no vocational sector has existed. In Finland, about 90,000 students 
strive for the first degree, called the Masters (Licentiate in medicine). Before 1981, 
a non-obligatory lower degree corresponding roughly to the Anglo-Saxon bachelor's 
degree was available in some fields. Admission is restricted: selection takes place 
mainly on the basis of success on the institutional entrance examinations. About half 
of the eligible secondary school leavers cannot be admitted. Higher education is state 
owned and financed and charges no tuition fees to students. Other study-related costs 
are supported by small grants and state guaranteed loans for a maximum of 7 years. 

United KingdoRi 

The United Kingdom has two main sectors of higher education, both of which are 
publicly funded: the universities and the polytechnics and colleges. This "binary" 
separation will cease in 1993 with the introduction of a single funding structure and 
the extension of the title of university to other institutions. Higher education courses 
are divided into three levels: postgraduate courses (with entry normally confined to 
graduates); first degree courses (mainly full-time for three to four years); and more 
vocationally-oriented sub-degree courses which include higher diploma and certificate 
qualifications (mainly part-time for two years). The polytechnics and colleges have 
developed missions which emphasise the provision of vocational studies at first degree 
level and below - and there is an emphasis on part-time and other courses designed 
to widen access and meet local and regional needs. In the universities the emphasis 
has continued to be towards full-time courses at first degree and postgraduate level, 
together with a broad research mission. Admission to higher education is selective, 
except for the Open University, and students on full-time first degree and diploma 
courees are entitled to a basic government grant covering tuition fees and maintenance. 
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In 1989 there were 1,067,000 students in British higher education; most (57%) enroled 
on first degree courses and a significant minority (30%) on sub-degree courses. 

7 South Africa 

The 21 South African universities are part of a system of tertiary (higher) education 
which also includes 16 technikons and 100 teacher training colleges. There are also 
129 technical colleges which are considered to be part of the post-secondary, but not 
tertiary system. Some of these institutions have similar characteristics to the British 
polytechnics while others correspond to the American community colleges. 
Technikons have selective admissions, requiring a matrio (ie a passing level on a 
national 12 grade examination) while most universities currently require metrio 
exemption (specified combinations of high school subjects combined with minimum 
levels of performance). Universities award degrees up to the doctorate level in most 
subject areas, while technikons award only diplomas up to the equivalent of a 
"technical doctorate" (the laureatus). The teacher training colleges mainly produce 
primary school teachers; universities and technikons are responsible for the training 
of secondary school teachers. Current student enrolments are as follows: universities 
- 320,000; Technikons - 90,000; teacher training colleges - 75,000, In the past, 
higher education has been characterised by racial segregation; the situation is, 
however, changing rapidly due to political changes in the country. 

The first academic degree 

In this proposed comparative study, student participation and persistence rates will be 
compared and analysed within the context of the most socially important degree within each 
society. These degrees will be referred to in this study as "first degrees". The research group 
recognises the fundamental differences of the first degrees in the countries of study. 

The basic characteristics of the first degree strongly reflect societal values long held in each 
country. Institutions in the United States, for example, have always been highly committed 
to student welfare and progress under the tradition of in loco parentis, whereas the German 
degree, on the other extreme, has been at least to some degree moulded around the 
Humboldtian concept of Lchr-und-Lcmfrcihcit (the freedom of teaching and learning). The 
first degrees also differ in approach and intent based on societal values. Whereas modem 
German professional study such as business economics has remained relatively theoretical and 
academic, professional studies in the United States have developed a more praxis-orientation. 
Further, while some first degrees such as those in the Netherlands and Germany lead to 
professional certification in subjects such as law, medicine and architecture, only a second, 
professional degree in the United States will ensure entrance into these professions. This 
section of the study will consist of a comparative analysis of the fundamental differences of 
the first degree based on the structural and philosophical elements which could have a 
substantial effect on the issue of student persistence. 

The comparative analysis 

The proposed study will examine longitudinal and similar data based on student cohorts 
available in each national system. The comparative analysis will examine characteristics of 
^ systems, proportion of students who complete each formal stage of education and the niamier 
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in which issues regarding student persistence are defined within the context of each national 
system. Student characteristics such as age, social economic status, gender, ethnic 
background and high school achievement will be described for each system. It was 
determined that the degree of detail associated with the longitudinal data varies from country 
to country. In the United States, for example, the data are very detailed, whereas in other 
countries, such as England, they may be less comprehensive. It was, therefore, decided that 
it would be necessary to first examine the data from all participating countries to determine 
comparable characteristics. It was also determined that longitudinal and similar data is 
available for the following countries: 



Austria Late 1070s 

Finland 1975/1985 

Germany 1979 

The Netherlands 1983 

South Africa 1980 

USA 1980 

UK 1987 



This study will supply policy makers and researchers a framework for understanding student 
participation and persistence in an international comparative context. This framework will 
also be useful in the development of future cross-national cohort studies for the participating 
countries. 



Time frame 

25 May 1992 

20 June 1992 

1 July 1992 
20 July 1992 



Information on longitudinal data to be sent to R Bijleveld by P 
Maatta, P Makae, G Parry, H Pechar, K Sclmitzer, and V Tinto 

Draft of research proposal sent to each participant by R 
Bijleveld 

Completed revisions sent back to R Bijleveld 
Complclcd proposal sent back to participants 



Funding 

Funding will be sought by each participant within his or her own country. In addition, overall 
"umbrella" funding will be sought where possible. Researchers may wish to begin funds 
earlier than the completed proposal deadline in July. 
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Working Group 4: The role of peer review in evaluation/assessment process 

The working group agreed that this project will be carried out in two stages. Stage 1 will 
consist of an inventory of forms of peer review in operation in a substantial number of 
countries and stage 2 will investigate the legitimacy of peer review. 

Stage 1: inventory of peer review processes 

The working group concluded that the inventory should: 

- be descriptive in nature; 

- include peer review processes and practices in the areas of both teaching and research; 

- identify peer review processes and practices at the various system levels eg national, 
regional, discipline, institutional; 

- be based on the dimensions identified in the Brennan paper, whilst acknowledging the effect 
that these dimensions could be further refined. 

As the costs involved with such an inventory would be relatively minor, no major problems 
were identified in terms of resourcing this part of the project. Possible sources of funding 
that were indicated include: the Dutch Ministry of Education and Science (action through 
CHEPS), the CNAA (action through Richard Lewis/John Brennan), and the American Council 
for Education (action through Elaine El-Khawas). 

Commitment to this stage of the project was expressed by members of the working group. 
The International Network for Quality Assurance Agencies in Higher Education set up by the 
Hong Kong Council for Academic Accreditation (HKCAA) was suggested as one of the 
possible in-roads to bodies actually engaged in review processes of teaching and research. 



Stage 2: investigation into the legitimacy of peer review 

Preliminary discussions were held to provide some reference for the design of this second 
phase. Although the term "legitimacy" proved to provide ample ground for discussion, it was 
agreed to use it as the heading for stage 2. It was suggested that: 

- detailed case studies in a limited number of countries would be the best method to address 
the question of legitimacy of peer review; 



- with a certain amount of variety included in the case studies based on the outcomes of the 
stage 1 inventory and certainly in teims of: 
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(a) peer review characterised by decisions embedded in the peer review itself, and 

(b) peer review processes whereby the actual decisions are taken on the basis of the outcomes 
and the review but are not actually pail of it. 

Several suggestions were made as to the focus of the project in terms of distinguishing 
characteristics such as (a) the objects of peer review; (b) the authority for peer review; (c) the 
historical context; and (d) the existence of "competing" forms of review, which partly refer 
back to the possible outcomes of stage 1 of the project. Based on the discussions in the 
group and the preliminary work done by John Brennan, Lynn Meek and John Brennan will, 
together develop the above ideas into a full project proposal. 
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